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^  This  paper  reports  preliminary  findings  from  an  investigation  of 
the  relationship  between  training  programs  for  AH-64  (Apache)  helicopter 
pilots  and  combat  mission  effectiveness.  Data  collection  and  analyses 
are  still  in  progress.  \ 


THE  RESEARCH  PROBLEM  AND  METHODS 

Two  key  questions  are  being  addressed  in  this  research. 
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How  can  Apache  aircrew  and  unit  training  be  designed  to 
improve  overall  combat  mission  effectiveness?^  fit  /U 

Is  it  possible  to  train  to  the  levels  of  proficiency 
needed  in  combat? 


One  reason  for  seeking  answers  to  these  questions  is  to  better 
understand  operational  capabilities  and  limitations  under  different 
training  programs.  This  was  the  primary  research  interest.  Another 
important  use  for  answers  to  these  questions  would  be  a  basis  for  making 
cost/benefit  tradeoff  analyses  (not  a  focus  of  the  present  research) . 

The  research  goal  was  to  causally  link  important  features  of 
training  programs  to  overall  mission  eifectiveness .  This  is  a 
formidable  task:  It  requires  assessing  technological  capabilities  and 
training  factors  at  both  the  individual  crew  and  unit  levels. 

Alternative  research  methods  were  considered,  among  them  field  studies 
and  computerized  simulation  models.  Large-scale  field  studies  were 
ruled  out  because  of  their  impracticality;  they  would  require  major 
restructuring  of  the  system  at  great  cost.5  Further,  field  studies  do 
not  afford  the  opportunity  to  assess  the  effects  of  training  with  new 

‘For  example,  a  major  training  feature  is  the  number  of  hours  per 
month  a  pilot  flies  his  aircraft  (the  AH-64  in  this  case)  .  In  order  to 
manipulate  this  training  feature,  different  hours  would  have  to  be 
selected  for  different  pilots  (or  units) .  Exploring  the  effects  of  this 
factor  alone  would  require  a  complete  restructuring  of  support  systems 
(e.g.,  maintenance)  that  are  geared  for  today's  training  hours;  among 
other  drawbacks,  restructuring  is  quite  costly.  And,  Number  of  Flying 
Hours  is  only  one  of  several  factors  that  need  assessment. 
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technology  (e.g.,  a  new  scout  helicopter  to  accompany  the  Apache  in  its 
missions,  such  as  the  OH-58D) .  However,  small-scale  field  studies  as 
well  as  studies  using  the  Apache  combat  mission  simulator  (CMS)  are 
still  being  considered  as  a  means  of  providing  additional  information  on 
the  effects  of  a  restricted  set  of  training  factors.  Computerized 
battle  simulation  models  were  ruled  out,  because  these  models  typically 
require  information  on  training  effects  as  inputs. 

Two  research  methods  are  being  used  in  concert  to  accomplish  the 
goal  of  causally  linking  training  to  combat  mission  effectiveness: 

-the  Subjective  Transfer  Function  (STF)  method2,  and 

-The  Algebraic  modeling  approach  to  subjective  measurement.3 


This  combination  of  research  methods  can  be  used  for  developing  causal 
models  of  activities  or  processes  of  complex  systems.  The  steps 
involved  in  the  STF  method  are  to 


-  identify  critical  factors  and  mission  outcomes  in 

conjunction  with  subject  matter  experts  (SMEs) ;  and 


-  structure  causal  hypotheses  (also  with  SMEs) . 

The  major  steps  in  the  algebraic  modeling  method  in  general  (with 
specifics  focused  on  this  research)  are  to 


2  Veit,  C.  T.,  and  Callero,  M.  Subjective  Transfer  Function 
Approach  to  Complex  System  Analysis.  The  RAND  Corporation,  R-2719-AF, 
March,  1981 . 

3  Example  references  are:  Anderson,  N.  H.  "Information  Integration 

Theory:  A  Brief  Survey,"  in  D.  H.  Krantz,  R.  C.  Atkinson,  R.  D.  Luce, 

and  P.  Suppes (eds . ) ,  Contemporary  Developments  in  Mathematical 
Psychology,  Vol.  2,  Academic  Press,  New  York,  1974;  Birnbaum,  M.  H. 

"The  nonadditivity  of  personality  impressions,"  Journal  of  Experimental 
Psychology  Monograph,  1974,  102,  543-561;  Krantz,  D.  H.,  and  A.  Tversky, 
"Conjoint-Measurement  Analysis  of  Composition  Rules  in  Psychology, " 
Psychological  Review,  1971,  78,  151-169;  Veit,  C.  T.,  "Ratio  and 
Subtractive  Processes  in  Psychophysical  Judgment,"  Journal  of 
Experimental  Psychology  (General),  1978,  107,  1,  81-107. 


(1)  Design  judgment  experiments  that  allow 

tests  of  hypothesized  effects  shown  in  the  STF  structure 
tests  among  algebraic  functions  hypothesized  a  priori  to 
measure  effects  found  in  the  data. 


(2)  Collect  judgement  data  from  SMEs 

(3)  Analyze  data  to  determine 

-  how  training  variables  affect  judged  mission  performance 

-  the  appropriate  algebraic  functions  to  measure  these  effects 

and  causally  relate  aircrew  and  unit  training  to  combat 
mission  effectiveness. 

The  algebraic  functions  are  theories  of  how  situational  information 
affects  respondents'  (SMEs  in  this  case)  judgments.  (For  example,  SMEs 
might  be  asked  to  judge  the  likelihood  that  their  battalion,  undergoing 
a  particular  training  program,  would  successfully  execute  its  close 
support  mission  in  Central  Europe.)  When  the  STF  approach  employs 
algebraic  modeling  methods,  conclusions  about  how  factors  affect 
outcomes  are  based  on  expert  judgment  and  tested  premises.  Both  of 
these  features  lend  credibility  to  resulting  conclusion.  It  is  the 
testability  feature,  however,  that  provides  the  scientific  base  for  the 
conclusions.  Effects  of  hypothesized  factors  and  predictions  of 
algebraic  functions  are  tested  and  rejected  if  not  supported  by  the 
data . 


THE  STF  TRAINING/EFFECTIVENESS  STRUCTURE 


In  the  STF  method,  factors  and  factor  levels  are  determined  through 
interaction  with  the  SMEs.  The  21  SMEs  who  participated  in  the  research 
reported  here  were  commanders,  operations  officers,  and  instructor 
pilots  from  3  corps  squadrons  and  2  division  battalions. 
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Figs.  1-5  describe  the  STF  training/effectiveness  structure.'1 
Figure  1  describes  the  complete  4-tiered  STF  training/effectiveness 
structure.  The  first  3  tiers  from  the  bottom  up  depict  the  factors 
hypothesized  to  affect  suboutcomes--crew  flight  proficiency,  crew 
mission  proficiency,  and  company  mission  effectiveness;  the  top  tier 
depicts  the  4  factors  hypothesized  to  affect  the  final  outcome- 
battalion  mission  effectiveness. 

Factor  levels  associated  with  the  factors  in  each  tier  are  shown  in 
Figs.  2-5.  The  bottom  tier.  Fig.  2,  hypothesizes  that  5  factors  are 
important  in  determining  crew  flight  prof iciency4 5--how  many  hours  per 
month  each  aircrew  member  flies  the  Apache,  how  many  NVS  and  CMS  flights 
they  have  per  month,  what  percent  of  the  time  they  train  as  a  crew,  and 
their  crew  experience.6 

Fig.  3  depicts  4  factors  hypothesized  to  affect  crew  mission 
proficiency:  flight  proficiency  on  the  ATM  tasks,  how  often  the  crew 
trains  on  a  particular  company  mission,  how  often  they  train  in  the 
combat  mission  simulator  (CMS)  on  that  mission,  and  the  type  of  mission. 
The  mission  type  could  be  close  support,  day  or  night,  in  the  forward 
area  against  armor,  deep  operations  (from  20  km.  to  100  km.  behind  enemy 
lines)  against  armor,  or  rear  operations  against  soft  targets. 

Fig.  4  hypothesizes  that  the  amount  of  training  with  the  company  on 
a  company  mission,  the  type  of  scout  that  accompanies  the  Apache  team  on 
the  mission, 7  and  the  median  crew's  mission  proficiency  (i.e.,  how  well 


4Acronyms  used  in  Figs.  1-9  are  as  follow: 

ATM:  Aircrew  Training  Manual 

NVS:  Night  Vision  System  (Apache) 

CMS:  Combat  Mission  Simulator 

5Crew  flight  proficiency  was  defined  as  the  percent  of  aircrew 
training  manual  (ATM)  tasks  the  crew  could  perform  almost  perfectly 
("with  the  grade  of  A")  on  a  given  check  ride. 

6A  senior  crew  member  was  defined  as  a  pilot  with  500  hours  or  more 
flying  time  in  the  Apache;  a  junior  crew  member  was  a  pilot  with  less 
than  this  number  of  total  flying  hours  in  the  Apache. 

7  The  LHX  is  armed,  has  state  of  the  art  optics,  an  automatic 
target  hand-off  system  (ATHS) ,  an  automatic  target  recognition  (ATR) 
system,  and  can  laser-designate  for  the  Apache's  Hellfire  missile.  The 
OH-58D  has  ATHS  and  laser  designation  capabilities;  the  OH-58C  has  none 
of  these  capabilities. 
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the  median  crew  performs  its  role  in  the  mission) 8  all  affect  a 
company's  performance  on  a  particular  mission. 

At  the  battalion  level  (Fig.  5),  the  median  company's  mission 
effectiveness  on  a  particular  mission,9  frequency  of  battalion  mission 
training  on  that  mission,  and  a  technological  factor,  threat  system 
location  accuracy, 10  are  all  hypothesized  to  affect  a  battalion's 
effectiveness  in  performing  the  mission. 

A  judgment  experiment  was  designed  for  each  tier  in  the  STF 
structure  using  the  factor  levels.  These  experimental  designs  generated 
situations  presented  to  participating  SMEs  for  their  judgments.11 

Fig.  1  notes  an  STF  (algebraic  function)  at  each  link  in  the 
structure.  When  the  4  appropriate  algebraic  functions  are  determined 
from  the  judgment  data,  they  serve  to  functionally  interlink  all  tiers 
of  the  structure;  the  final  structure  and  its  associated  composite 
function  is  an  effectiveness  model  for  evaluating  training  programs.  An 
existing  or  proposed  training  program  can  then  be  described  in  terms  of 
factor  levels  in  the  structure  and  the  composite  function  computed  to 
provide  a  prediction  of  battalion  mission  effectiveness  for  that  program 
(as  well  as  a  predicted  outcome  at  each  tier).12  For  all  factors  with 
physical  values  as  factor  levels  (e.g.,  number  of  NVS  flights  per  month, 

8There  are  5  AH-64s  in  a  heavy  attack  team.  A  median  crew 
proficiency  of  80%  states  that  half  of  the  crews  perform  at  a  higher 
mission  proficiency  level  than  80%  and  h=>lf  at  a  lower  level. 

9  There  are  3  companies  in  an  attack  helicopter  battalion. 

10  Threat  system  location  accuracy  refers  to  the  accuracy  with 
which  the  location  of  the  threat  systems  (including  the  targets)  are 
known  to  the  battalion  and  passed  to  the  companies. 

nThe  nature  of  the  experimental  designs  used  for  the  4  different 
judgment  experiments  was  driven  by  the  -nature  of  the  factors  and  the 
algebraic  functions  entertained  as  STFs.  Factorial  combinations  of 
factor  levels  was  the  basic  design  feature.  However,  this  design 
feature  was  extended  so  as  to  be  able  to  test  among  a  wider  variety  of 
algebraic  functions.  More  details  on  experimental  designs  is  presented 
in  Veit,  C.  T.,  M.  Callero,  and  B.  J.  Rose,  "Introduction  to  the 
Subjective  Transfer  Function  Approach  to  Analyzing  Systems,  The  Rand 
Corporation,  R-3021-AF,  1983. 

12The  STF  is  called  a  transfer  function,  because  when  the  composite 
function  is  used  in  analyses,  the  output  of  a  function  lower  in  the 
hierarchy  is  transferred  as  an  input  factor  level  to  the  function  at  the 
next  highest  level  in  the  hierarchy. 
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frequency  of  training,  threat  location  accuracy) ,  interpolated  and 
extrapolated  values  describing  a  proposed  training  program  or  capability 
level  can  be  used  in  the  effectiveness  analysis.13 

RESULTS  OF  DATA  ANALYSES  FOR  THE  CLOSE  SUPPORT  DAY  MISSION 

Data  collected  to  date  for  the  close  support  day  mission  are 
reported  in  this  section.  Data  collection  and  model  analyses  are 
ongoing . 

Figs.  6-9  are  graphs  of  the  more  salient  features  of  the  close 
support  data.  In  Fig.  6,  judged  percent  of  ATM  tasks  performed 
"perfectly"  (at  a  top  level)  is  plotted  on  the  y-axis.  The  training 
program  is  described  in  terms  of  the  number  of  CMS  flights  per  month 
(plotted  on  the  x-axis) ,  number  of  NVS  flights  per  month  (plotted  as 
separate  curves) ,  and  the  total  number  of  Apache  flying  hours  per  month 
(separate  panels) .  These  data  are  for  a  crew  made  up  of  1  junior  and  1 
senior  member  who  train  together  80%  of  the  time. 

As  can  be  seen,  all  of  the  curves  are  higher  in  the  right-hand 
panel  of  Fig.  6,  indicating  that  increasing  total  flying  hours  from  10 
to  20  per  month  increases  proficiency  for  any  comparable  CMS  and  NVS 
combination.  Horizontal  lines  drawn  through  the  curves  provide 
information  about  tradeoffs  among  the  training  factors  in  affecting  ATM 
task  proficiency.  An  interesting  tradeoff  can  be  seen  from  the  data 
points  designated  by  squares.  If  pilots  get  1  NVS  flight  every  other 
month  (bottom  curve  in  both  panels) ,  the  same  performance  can  be 
achieved  by  increasing  the  number  of  CMS  flights  to  4  per  month  as 
increasing  total  Hying  hours  to  20  per  month  (and  keeping  number  of  CMS 
flights  at  1) .  The  triangulated  data  point  indicates  that  this  same 
level  of  proficiency  can  be  achieved  by  keeping  total  flying  hours  at  10 
per  month,  decreasing  CMS  flights  from  4  to  2  per  month  and  increasing 
number  of  NVS  flights  to  once  a  month.  The  reader  can  pick  out  many 
other  interesting  tradeoffs  in  these  data.14 

13  Interpolated  and  extrapolated  values  can  be  appropriately  used 
in  analyses  because  the  STFs  are  tested  theories  of  experts' 
perceptions;  it's  the  theoretical  base  that  permits  such  extensions. 

The  theories  have  undergone  rigorous  tests;  numerous  vying  theories  have 
been  rejected  because  the  data  did  not  support  their  unique  predictions. 

14As  mentioned  earlier,  model  analyses  are  incomplete.  However, 
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In  Fig.  7,  crew  proficiency  level  is  plotted  on  the  x-axis  with  a 
separate  curve  for  the  number  of  CMS  flights  per  month  and  a  separate 
panel  for  the  number  of  times  a  month  the  crew  trains  with  the  company 
on  the  close  support  mission.  The  squared  data  points  indicate  that, 
for  a  crew  at  the  50%  ATM  task  proficiency  level,  it  is  possible  to 
achieve  the  same  crew  mission  proficiency  level  in  a  program  that  trains 
with  the  company  once  a  month  as  3  times  a  month,  if  CMS  training  is 
increased  from  once  to  twice  a  month.  This  implies  that  CMS  training  is 
a  very  valuable  tool  for  mission  training.  Two  advantages  to  CMS 
training  over  training  in  the  field  is  that  pilots  can  really  shoot  at 
and  laser  designate  targets. 

In  Fig.  8,  frequency  of  company  mission  training  is  plotted  on  the 
x-axis;  a  separate  curve  is  for  each  scout  helicopter  type,  and  a 
separate  panel  is  for  the  different  levels  of  median  crew  performance. 
Clearly,  having  the  most  advanced  scout  helicopter  (the  light  attack 
helicopter,  LHX)  and  training  on  the  mission  frequently  (3  times  a 
month)  produces  the  greatest  mission  effectiveness,  as  would  be 
expected.  The  squared  data  points  in  the  right-hand  panel  indicate  that 
a  company  with  a  median  crew  mission  proficiency  level  of  80%  would  be 
expected  to  perform  its  company  mission  equally  well  in  the  following  2 
training  programs:  (a)  a  program  that  trained  the  crews  on  the  company 
close  support  mission  only  once  every  other  month,  but  the  Apache  team 
was  accompanied  by  OH-58D  scouts,  or  (b)  a  program  that  trained  the 
crews  on  the  mission  3  times  a  month  but  the  Apache  team  had  no 
accompanying  scouts.15 

The  triangulated  data  points  in  the  left-hand  panel  of  Fig.  8 
indicate  that  crews  performing  at  the  50%  level  do  just  as  well  with  an 
OH-58D  as  an  LHX  if  they  increase  their  mission  training  from  once  every 

when  the  STF  has  been  determined  for  these  data,  tradeoffs  can 
appropriately  be  made  among  interpolated  and  extrapolated  values.  These 
data  most  likely  closely  approximate  predicted  values  from  the 
theoretical  model,  since  deviations  of  predicted  from  obtained  values 
are  typically  small. 

IJWhen  an  Apache  team  flies  autonomously,  the  mission  takes  on  the 
additional  task  of  locating  their  targets  and  air  defense  threats. 
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other  month  to  once  a  month.  They  can  do  about  the  same  with  an  OH-58C 
if  they  increase  their  mission  training  to  3  times  a  month  (diamond  in 
left-hand  panel) . 

Figure  9  depicts  the  direct  tradeoffs  that  affect  battalion  mission 
effectiveness.  Recall  from  Fig.  1  that  the  factor.  Median  Company 
Performance  (separate  panels) ,  is  affected  by  all  the  factors  in  the  3 
tiers  of  the  structure  below  this  top  Battalion  Mission  Effectiveness 
tier.  Frequency  of  battalion  training  is  plotted  on  the  x-axis  in  Fig. 
9,  with  a  separate  curve  for  each  level  of  threat  location  accuracy. 

The  squared  data  points  in  the  two  panels  of  Fig.  9  illustrate  a 
t raining/technology  tradeoff.  A  battalion  with  a  median  company 
performance  level  of  50%  (left-hand  panel)  requires  a  target  location 
accuracy  capability  of  500  meters  to  perform  as  well  as  a  battalion  with 
a  median  company  performance  of  80%  that  uses  a  poorer  target  location 
accuracy  device  (1000  meters).  Comparison  of  the  triangulated  data 
points  in  the  2  panels  of  Fig.  9  illustrates  another  training/technology 
tradeoff.  If  training  is  fairly  poor  (i.e.  mission  training  is  only 
once  every  other  month  and  previous  training  resulted  in  a  median 
company  effectiveness  of  50%) ,  a  better  target  location  accuracy  (within 
1000  meters)  is  needed  to  achieve  comparable  battalion  mission 
effectiveness  with  a  better  performing  battalion  (mission  training  is 
once  a  month  and  previous  training  resulted  in  a  median  company 
effectiveness  of  80%)  using  a  poorer  target  location  accuracy  capability 
(3000  meters) . 

CONCLUDING  REMARKS 

Numerous  tradeoffs  like  those  described  above  can  be  seen  in  these 
data.  Knowledge  of  the  tradeoffs  provides  information  about  the 
effectiveness  of  alternative  training  programs.  At  the  present  time, 
the  Army  is  unlikely  to  increase  the  number  of  flying  hours.  These  data 
indicate  it  is  possible  to  improve  proficiency  of  aircrews  and 
operational  effectiveness  of  units  within  the  current  flying-hour 
program.  Also,  it  is  evident  from  the  scout /t ra ining  tradeoffs  seen  in 
Fig.  8  and  the  threat  location  accuracy/training  tradeoffs  seen  in  Fig. 
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9  that  increases  in  training  can  provide  comparable  or  greater 
effectiveness  increases  than  technological  improvements.  This  is  not  to 
say  that  the  Army  shouldn't  purchase  advanced  technology.  Mission 
effectiveness  improvements  are  greatest  with  improvements  in  both  areas. 

When  completed,  the  effectiveness  model  shown  in  Fig.  1  can  be  used 
as  a  guide  for  changing  training  programs.  Major  considerations  to  such 
changes  would  be  cost,  manpower,  and  local  constraints  such  as  training 
areas  and  restrictions  on  night  flying. 

An  interesting  extension  to  this  project  would  be  a  cost/benefit 
analysis  where  costs  could  be  manipulated  and  benefits  could  be  judged 
by  SMEs  in  a  framework  like  this  that  supports  causal  conclusions. 
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Fig.  1 — STF  Structure  of  Causal  Hypotheses 
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Fig.  2— STF  Crew  Flight  Proficiency  Structure 
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1/  2  months 

50% 

OH-S8C 

Deep  Operations 

Rear  Operations 

1 

1 

1 

NONE 

Fig.  4 — STF  Company  Mission  Effectiveness  Structure 


BATTAUON  MtSSlON 
EFFECTIVENESS 
(%  successful) 


PERCENT  of  time  battalion  would  successfully  pe 


form  the  mission  (survive  and  engage  targets) 


STF 


MISSION 

(iyp®) 

BATTALION  MISSION 
TRAINING 
(number/month) 

MEDIAN  COMPANY’S 
MISSION 
PROFICIENCY 
(%  successful) 

THREAT  SYSTEM 
LOCATION 
ACCURACY 
(meters) 

Close  Support; 

3/month 

100% 

500m 

(Day;  Armor) 

1 /month 

80% 

1000m 

Close  Support; 

(Night;  Armor) 

Deep  Operations 

Rear  Operations 

1/  2  months 

50% 

1 

1 

1 

3000m 

Fig.  5 — STF  Battalion  Mission  Effectiveness  Structure 
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EXPERIENCE  LEVEL  OF  PILOTS:  1  JUNIOR;  1  SENIOR 
PERCENT  OF  TRAINING  AS  A  CREW:  80% 


Fig.  6 — Crew  Flight  Proficiency 


Likelihood 

Etlective 

Performance 

(crew) 


Fig.  7 — Crew  Mleelon  Proficiency 
(Cloee  Support,  Dey) 
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Likelihood 

Effective 

Mission 

(Company) 


Median  Crew  Perlormence:  50%  Median  Crow  Parformance:  80% 


Frequency  of  Company  Training  on  the  Mission 


Fig.  8 — Company  Mission  Effectiveness 
(Close  Support,  Day) 


Likelihood 

Effective 

Mission 

(Battalion) 


Median  Company  Effectiveness:  50%  Median  Company  Effectiveness:  80% 


Frequency  of  Battalion  Training  on  the  Mission 


Fig.  9 — Battalion  Mission  Eftactlvaneaa 
(Close  Support,  Day) 


